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Abstract. This paper presents an extension of the Additive Factors Model to 
predict learning for students by accounting for aspects of collaboration. The re- 
sults indicate that student performance is predicted more accurately when the 
model includes parameters that capture influences of working collaboratively. 
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A strength of intelligent tutoring systems (ITSs) is that they can be modified through 
offline student modeling to provide better instruction for students. Although ITSs 
have been shown to support students working in groups [2], the statistical models that 
are used to refine and support ITSs often do not take into account features of collabo- 
ration (e.g., partner knowledge). Student modeling might be improved and learning 
might be supported even better, if we took into account collaborative features. Thus, 
we extended the Additive Factors Model (AFM), which is a logistic regression model 
frequently used in offline analyses of ITSs [1]. The standard AFM [1] calculates the 
log-odds that a given student correctly solves a given step in a problem as a function 
of three estimated parameters that capture the student’s initial proficiency, the ease of 
the skills involved in the step, and the learning rates for those skills. We modified the 
standard AFM so that the model has separate learning rates depending on if a skill is 
being learned in an individual or collaborative environment (AFM+C) since the learn- 
ing processes may differ. Further, to better understand how a student’s partner’s 
knowledge may impact the prediction of a student’s learning, we analyzed four differ- 
ent variations to take into account partner knowledge: partner pretest score classified 
as low/average/high (AFM+PPS), absolute difference between student’s and partner’s 
pretest scores classified as homogeneous or heterogeneous (AFM+AD), and two di- 
rectional differences between student’s and partner’s pretest scores including low- 
er/similar/higher (AFM+DD) and lowest/lower/similar/higher/highest (AFM+LD). 
We hypothesized that the models with collaborative/individual learning rates and 
the models with partner knowledge would be a better fit than the standard AFM. We 
used two datasets consisting of log data from conceptually or procedurally-oriented 
ITSs. In each data set, students were working individually or collaboratively. We 
measured the accuracy with which the models predicted student performance for both 


datasets using log likelihood, Akaike information criterion (AIC), and Bayesian in- 
formation criterion (BIC). The log likelihood does not take into account the complexi- 
ty of the model while the AIC and BIC do account for the complexity of the model. 


Table 1. Prediction accuracy for all models. The asterisks mark the best performing model 
while a plus sign indicates that the model performed worse than the baseline for that measure. 


Model Name Log Likelihood | AIC BIC Comparison to 
Standard AFM 

Conceptually-oriented ITS 

Standard AFM _ | -8769.7 17549.3 | 17589.1 

AFM+C -8731.2* 17478.3* | 17542.0* | N°(3)=77.0, p<0.001 

AFM+PPS -8759.3 17534.6 | 17598.2+ | N°(3)=20.8, p<0.001 

AFM+AD -8768.6 17553.3+ | 17616.9+ | N°(3)=2.1, p=0.56 

AFM+DD -8761.2 17538.4 | 17602.1+ | N°(3)=16.9, p<0.001 

AFM+LD -8760.5 17536.9 | 17600.6+ | N°(3)=18.4, p<0.001 

Procedurally-oriented ITS 

Standard AFM _ | -7991.3 15992.6 | 16032.0 

AFM+C -7942.8* 15901.5* | 15964.5* | N°(3)=97.1, p<0.001 

AFM+PPS -7989.0 15994.0+ | 16057.1+ | N°(3)=4.6, p=0.20 

AFM+AD -7989.2 15994.4+ | 16057.4+ | N°(3)=4.2, p=0.24 

AFM+DD -7988.7 15993.5+ | 16056.5+ | N°(3)=5.2, p=0.16 

AFM+LD -7987.9 15991.9 | 16054.9+ | _N7(3)=6.8, p=0.08 


The models with collaborative/individual learning rates were a better fit than the 
standard AFMs as shown in the AFM+C rows of Table 1. The models with variations 
of the partner’s pretest were a better fit only for the conceptually-oriented data as 
shown in the comparison column in Table |. This may be caused by different types of 
talk occurring around conceptual and procedural knowledge with a partner, which 
may have an influence on learning. Overall, our results show that by including collab- 
orative features within a model, we can improve the learning prediction. With a more 
accurate learning prediction for an ITS, in future work, we may be able to better re- 
fine the instructional support used in individual and collaborative ITSs. 
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